Skip to content

[pylint] Fixes all use-maxplit-args, consider-using-enumerate #12172

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 27, 2024

Conversation

Pierre-Sassoulas
Copy link
Member

Using maxsplit=1 from use-maxplit-args will make splits slightly faster and is the important one that will improve performances.

The other fix are about readability / style.

@Pierre-Sassoulas Pierre-Sassoulas marked this pull request as draft April 1, 2024 07:06
@Pierre-Sassoulas
Copy link
Member Author

Converting to draft as I need to add tests for the missing coverage

@@ -207,7 +207,7 @@ def test_visit_norecurse(self, path1):

@pytest.mark.parametrize(
"fil",
["*dir", "*dir", pytest.mark.skip("sys.version_info <" " (3,6)")(b"*dir")],
["*dir", "*dir", pytest.mark.skip("sys.version_info < (3,6)")(b"*dir")],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this:

Suggested change
["*dir", "*dir", pytest.mark.skip("sys.version_info < (3,6)")(b"*dir")],
["*dir", "*dir", b"*dir"],

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right ! Removed the reminder for the whole codebase in 7ded8d8

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yikes, we had skip instead of skipif and the tests where broken

Copy link
Member

@RonnyPfannschmidt RonnyPfannschmidt Apr 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

965e942 introduced the issue replacing a pylib basestring with just str

wrong ref used

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for looking into it, I was stuck. I'm not sure I understand though, should I replace ["*dir", "*dir", b"*dir"], back to ["*dir", "*dir", pytest.mark.skipif("sys.version_info < (3,6)")(b"*dir")], and then fix the tests ? Maybe in another merge request at this point ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the Visitor class of the local path clone, there is a condition we need to replace
https://github.com/pytest-dev/pytest/blob/main/src/_pytest/_py/path.py#L140 - it should also check for bytes

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when asottile took out a lot of cruft a basestring usage was misinterpreted and replaced with just str

unfortunately the skips where wrong and thus the error was never noted as the tests skipped instead of passing

Copy link

@ferdnyc ferdnyc Jul 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that's where things went wrong.

I checked the py code, and in Python 3.0+ py.basestring was indeed set equal to str. The problem is that using str and bytes interchangeably isn't valid in Python 3, and py.path.local never fully accounted for that.

Even in the upstream, this happens in Python 3.6 (or higher, I'm sure):

>>> import py
>>> dir = py.path.local(b'/tmp')
>>> dir
local(b'/tmp')
>>> dir.strpath
b'/tmp'
>>> dir.parts()
[local(b'/'), local(b'/tmp')]
>>> dir.basename
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../venv3.6/lib64/python3.6/site-packages/py/_path/common.py", line 141, in basename
    return self._getbyspec('basename')[0]
  File ".../venv3.6/lib64/python3.6/site-packages/py/_path/local.py", line 293, in _getbyspec
    parts = self.strpath.split(self.sep)
TypeError: a bytes-like object is required, not 'str'
>>> 
>>> # Part of the problem being...
>>> dir.sep
'/'
>>> dir.bestrelpath(b'/')
"b'/'"
>>> # ^^^ Note that that's a _string_ containing "b'/'",
>>> # IOW the result of calling str() on a bytes object

...That the tests were being unconditionally skipped, and never caught that, is the bigger problem. But fixing bytes support will not be easy as _pytest/_py/path.py (like py.path.local before it) is very sloppy with how it handles inputs. When passed bytes, things will sorta work... until they very suddenly really don't.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, and just for fun, I found two more instances of tests being unconditionally skipped by pytest.mark.skip(), instead of conditionally with .skipif(). Just in that one test file.

@pytest.mark.skip("sys.version_info < (3,6)")
def test_fspath_open(self, path1):
f = path1.join("opentestfile")
open(f)
@pytest.mark.skip("sys.version_info < (3,6)")
def test_fspath_fsencode(self, path1):
from os import fsencode
assert fsencode(path1) == fsencode(path1.strpath)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the report; I've opened a new issue to warn when that kind of thing happens 🙏

@Pierre-Sassoulas Pierre-Sassoulas changed the title [pylint] Fixes all use-maxplit-args, consider-using-enumerate, implicit-str-concat [pylint] Fixes all use-maxplit-args, consider-using-enumerate May 21, 2024
Copy link
Member

@Zac-HD Zac-HD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks Pierre!

If there are any follow-ups, we'll be happy to take them in another PR 🙂

@Zac-HD Zac-HD marked this pull request as ready for review May 27, 2024 19:17
@Zac-HD Zac-HD merged commit 88fae23 into pytest-dev:main May 27, 2024
27 checks passed
@Pierre-Sassoulas Pierre-Sassoulas deleted the more-pylint-fixes branch May 27, 2024 19:42
@Pierre-Sassoulas
Copy link
Member Author

Thank you @Zac-HD, I was wondering how to cover the line that was not covered but if it's mergeable as is for you I'm going to stop wondering. I'll open a follow-up regarding the problem raised by the implicit-str-concat 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants